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Abstract 

We propose necessary and sufficient conditions for a sensing matrix to be "s-semigood" ~ to 
aliow for exact ^i-recovery of sparse signals with at most s nonzero entries under sign restrictions 
on part of the entries. We express error bounds for imperfect £i-recovery in terms of the 
characteristics underlying these conditions. These characteristics, although difficult to evaluate, 
lead to verifiable sufficient conditions for exact sparse £i-rccovery and thus efficiently computable 
upper bounds on those s for which a given sensing matrix is s-scmigood. Wc examine the 
properties of proposed verifiable sufficient conditions, describe their limits of performance and 
provide numerical examples comparing them with other verifiable conditions from the literature. 

1 Introduction 

Assessing a sparse signal from an observation has been one of the main research areas in Compressed 
Sensing and sparse signal recovery. In practice, a priori information about the signal to be recovered 
often exists and will be beneficial if taken into account in the recovery procedure. In this paper, we 
suppose that the a priori information about a sparse signal w € amounts to the sign restrictions, 
and is given as the subsets P+ and P_ of {1, P+ H P_ = 0, such that Wi > for i £ P+ 
and Wi < for i £ P_. Therefore we address the following recovery problem: given an observation 



y = Aw + e, (1) 



'Research of the second and the third authors was supported by the Office of Naval Research grant # 
N000140811104. 
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where A G R^^x"- (in this context m < n) is a given matrix, e G R"^ is the observation error, assess 
a sparse signal w G R" satisfying sign restrictions. 

A celebrated solution to the problem is given by the £i-recovery, which amounts to taking, as 
an estimate of w, an optimal solution w to the optimization problem 

w G Argmin^. {||a;||i : \\Ax - y\\ <e, > Vz G P+, Xj < Vi G P-} (2) 

(here e is an a priori bound on the norm ||e|| of the observation error, || • || being some norm on 
R™"). When there are no sign restrictions (i.e. P+ = P_ = 0), we arrive at the estimator playing 
the central role in the Compressive Sensing theory. The central result here is that when signal w 
is s-sparse (i.e., with at most s nonzero entries) and the matrix A possesses a certain well-defined 
(although difficult to verify) property, then the ^i-recovery Hi is close to w, provided the error bound 
e is small (for a comprehensive survey see [4] and references therein). Our goal here is to propose 
efficiently verifiable sufficient conditions on A which allow for similar 'consistency" results, with 
emphasis on the case where sign restrictions are present. 

To outline our results and to position them with respect to what is already known, let us start 
with noiseless recovery (i.e., e = and y = Aw). Here we are interested to answer the question: 

Whether A is such that whenever the true signal w in (f ) is s-sparse and satisfies 
the sign constraints Wi > 0, i £ P-f , Wi < 0, i G P_, the ii-recovery 

w G Argmin^,. {||x||i : Ax = y, Xj > Vi G P+, Xj < Vi G P_} (3) 

recovers w exactly. 

If the answer is positive, we say that A is s-semigood^ . 

The theory of Compressive Sensing provides several sufficient /necessary and sufficient conditions 
for the ^i-recovery to be exact. For example, when no sign constraints are imposed on w, Donoho 
and Huo [!)] prove that A is s-good if for any set I C {1, ...,n} of cardinality < s it holds 

\zi\ < \ zi\ for any z G KerA. (4) 

This condition has been extensively investigated. Its necessity has been established in [8]; it has 
been discussed in [16, 18] (under the name of strict s-balancedness), where its link to the geometric 
necessary and sufficient condition of s-goodness from [II] has been discussed. In [G], this condition 
has was also related to the sufficient condition ( ^^Null Space Property^^ ) for successful combinatorial 
recovery. 

The first characterization of s-semigoodness for the case when w is nonnegative (i.e. P+ = 
{I, ...,n}) was proposed in the founding paper of Donoho and Tanner [10] in terms of neighboring 
properties of the polytope AS, S being the standard simplex 5 = {x G R" : x > 0, Xj < I}. 
This paper contains also several important examples of m x n matrices which are [yj-semigood 
(here [aj stands for the integer part of a) and demonstrates that various types of randomly gen- 
erated matrices possess this property with overwhelming probability. Extending the results from 
Donoho and Huo [9], an equivalent characterization of s-semigoodness has been provided in the 

^We use the term "s-semigoodness" to comply with the terminology of the companion paper [14], where we used 
the name s-goodness to indicate that ^i-recovery as in (3) without the sign restrictions is exact. 
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nonnegative case by Zhang in [17, 18], where it is shown that A is s-semigood if and only if the 
kernel of A, KevA, is strictly half s-halanced, meaning that for any set / C {1, ...,n} of cardinality 
< s it holds 



y Zi < \ for any z £ KerA such that Zj < 0, for all i ^ I. 



(5) 



It should be mentioned that the necessary and sufficient conditions for s-semigoodness from 
(4), (5) and [10, 11] share a common drawback - they seemingly cannot be verified in a com- 
putationally efficient way. To the best of our knowledge, the only efRciently veriEable conditions 
for s-semigoodness offered by the existing Compressive Sensing theory are the sufficient conditions 
based on the mutual incoherence 



where Ai are columns of A (assumed to be nonzero). Clearly, the mutual incoherence can be easily 
computed even for large matrices. Unfortunately, it turns out that that the estimates of "level of 
(semi)goodness" of a sensing matrix based on mutual incoherence usually are too conservative, in 
particular, they are provably dominated by the verifiable Linear Programming (LP) based sufficient 
conditions for s-goodness proposed in the companion paper [14] and based on characterization of 
s-goodness given in (4). Another verifiable sufficient condition for s-goodness, which uses the 
Semidefinite Programming (SDP) relaxation, has been recently proposed in [7]. 

The contributions of this paper, which follow the approach developed in [14], are as follows. 

1. Taking existing characterizations of (semi)goodness (4), (5) as a starting point, we develop in 
Section 2, several equivalent necessary and sufficient conditions for s-semigoodness of a matrix 
A in the case of general-type sign restrictions. Then in Section 3, we establish error bounds 
for inexact ^i-recovery (noisy observation (1), imprecise optimization in (2), nearly-sparse 
true signals); these bounds are expressed in the same terms as the necessary and sufficient 
conditions for s-semigoodness from Section 2. These bounds can be seen as an extension to 
the sign restricted case of bounds of Section 3 in [14] and as a special case of the bounds 
provided in Theorem 4.1 of []>>]. To the best of our knowledge, these bounds that incorporate 
sign information of the signal are new. 

2. The major goal of this paper is to use the LP relaxation techniques from [14] to derive novel 
efficiently verifiable sufficient conditions for s-semigoodness. These conditions allow one to 
build, in a computationally efficient fashion, lower bounds on the "level of s-semigoodness" of 
a given matrix A, that is, on the largest s = s^{A) for which A is s-semigood with respect to 
given P±. Some properties of these verifiable conditions, same as limits of their performance, 
are studied in Sections 4, 5, where we provide also a computationally efficient scheme for 
upper bounding of s*(A). In Section 6, we develop another efficiently computable lower 
bound for s*(A) by applying the SDP relaxation, similar to the approach developed in [7] 
for the "unsigned" case P± = 0. In Section 7 we report on numerical experiments aimed 
at comparing the "power" of our LP-based sufficient conditions for s-semigoodness, their 
"unsigned" prototypes from [14], and conditions based on mutual incoherence. We show that 
incorporating the sign information can improve the bounds on the level of s-semigoodness, 
and that the bounds based on LP relaxations clearly outperform the bounds based on mutual 
incoherence. 



IJ,{A) = max 




(6) 



ij^j AfAi 
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3. It turns out that our verifiable sufficient conditions for s-semigoodness can be expressed in 
terms of specific properties of the linear recovery w^^^ = Y^y associated with an appropriate 
mx n matrix Y. In Section 8, we propose and justify a new non-EucUdean Matching Pursuit 
algorithm associated with this linear recovery. 



2 Necessary and sufficient conditions for s-semigoodness 

Let ^ be an m X n matrix, let s, 1 < s < m, be an integer, and let P+, P_ and P„ be a partition 
of {1, . . . , n} into three non-overlapping subsets. We say that A is s-semigood, if for every vector w 
with at most s nonzero entries satisfying Wi >0 for i € P^, and Wi <0 for i € P_, w is the unique 
optimal solution to the problem 

Opt = min{||z||i : Az = Aw, > Vi E P+, < Vi G P_} . (7) 

z 

Our primary goals are to find necessary and sufficient and veriBahle sufficient conditions for A 
to be s-semigood. 

Note that without loss of generality we may assume P_ = 0. Indeed, by replacing the partition 
P+, Pn with the partition P+ = P+ U P_, P_ = 0, P„ = P„, and matrix A - with the matrix 
A obtained from A by multiplying the columns with indices i € P_ by —1, s-semigoodness of A 
with respect to the original sign restrictions given by P±,Pn is equivalent to the s-semigoodness 
of the new matrix A with respect to the new sign restrictions. By this reason, we assume from 
now on that P^ = 0. Besides this, we assume without loss of generality that P+ = {1, ■■■,p} and 
Pn = {p + 1) ^} for some p. From now on, we denote by Vn the set of all signals satisfying the 
sign restrictions: 

rn = {w eiC -.wtyo \fi e P+}. 

Note that since P_ = 0, (7) simplifies to 

Opt = min{||z||i : Az = Aw, Zi > Vi G P+} . (8) 

z 

Let us fix a norm || • || on R", and let || • ||* be the conjugate norm. 

Proposition 2.1 Let m,n,s and P^ be given. The following six conditions on an m x n matrix 
A are equivalent to each other: 

(i) A is s-semigood; 

(ii) For every subset J of {1, ...,n} with Card(J) < s, and any x € Ker^\{0} such that x,, < 
for all i G P4. \ J one has 

i£jnP+ ieJnPn i^J 

(iii) There exists ^ G (0, 1) such that for every subset J of {1, ...,n} with Card(J) < s and any 
X £ Ker^ such that Xi <0 for all i G P-f \ J one has 
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(iv) There exist ^ G (0, 1) and 9 G [1, oo) such that A satisfies the condition SGs(.^, 9) as follows: 
for every x G KerA and every subset J o/{l, ...,n} with Card(J) < s, one has 

i£jr\P+ if^JnPn \iePn\J ieP+\J / 

or, equivalently: for all x G Ker^, Q{x) < ^^'(x) where 

(9) 



Q{x) := j^^ax^^ [EiGJnP+ max[(l - ()x^, (1 + 9^)xi] + EieJnP„(l + Ok 

Card( J)<s 



(v) There exist ^ G (0,1), G [l,oo) and j3 G [0, oo) such that A satisfies the condition 
SGs^/3(^, 9) as follows: 

for every x G R" and every subset J o/{l, ...,n} with Card(J) < s, one has 

^ Xi+ \xi\<l3\\Ax\\+i{ \xi\+ Y V'(a;i) , -(/-(t) = max[-t,0t]. 

jeJnP+ i£JnP„ \i&Pn\J ieP+\J / 

(vi) There exist ^ G (0,1) and /3 G [0, oo) suc/i that A satisfies the condition SGg^/si^,) as follows: 
for every J C {1, n} with Card( J) < s and any x G R" such that Xi < for all i G P+ \ J, one 
has 

J]] Xi+ ^ \xi\ < l3\\Ax\\ + CY\^i\- 
ieJnP+ ieJnPn i^J 

We provide the proof of Proposition 2.1 in Appendix A. 

As we have already mentioned in Introduction, when = or P-f = 0, the characterizations 

(i) -(iv) of s-semigoodness are not completely new. For instance, when P„ = 0, a necessary and 
sufficient condition for s-semigoodness of A in the form (ii) has been established in [17] (compare 

(ii) to the definition (5) of half s-balancedness of KerA). On the other hand, the equivalent 
formulation of this characterization in terms of conditions SGg, i3{£,, 9) and SGs./3(i^) seems to be 
new. We are about to demonstrate that the latter two conditions allow to control the error of 
^i-recovery in the case when the vector w G R"' is not s-sparse and the problem (8) is not solved 
to exact optimality. 



3 Error bounds for imperfect ^i-recovery 

We have seen that the conditions provided in Proposition 2.1 are responsible for s-semigoodness of 
a sensing matrix A, that is, for the exactness of i?i-recovery in the "ideal case" when the true signal 
w is s-sparse, there is no observation error, and the optimization problem (8) is solved to exact 
optimality. Below we demonstrate that these conditions control also the error of £i-recovery in the 
case when the signal w G Vn is not exactly s-sparse, there is observation noise and problem (8) is 
not solved to exact optimality. The corresponding error bound (cf [14, Proposition 3.1, Theorem 
3.1]) is as follows: 
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Proposition 3.1 Let w G Vn be such that \\w—w^\\i < ^i, where is the vector obtained from w by 
replacing all but the s largest in magnitude entries in w with zeros, let y be such that \\Aw — y\\ < e, 
and let, finally, x be an approximate solution to the optimization problem 

Opt = min{||z||i : \\Az - y\\ < e, Zi > Vi e P+} . (10) 

z 

such that \\x\\i < Opt + u and \\Ax — y\\ < 6. 

1. If A satisfies the condition SGg, i3{(,,0) with some ^ € (0,1), /? € [0, oo) and 9 € [l,oo), then 

1 + e , 2(1+ eg) ^ 2/3 

\\X - W\\l < -^—^-^ + ^+Y3^(g + '^)- (11) 

2. If A satisfies the condition SGs^p{£,) with some ^ £ (0,1), (3 G [0, oo), then 

1 + f 2(1 + /3a) 2/3 , 
||x - wh < Y^l^ + > + 13^^^^ ^' ^^^^ 

where a stands for the maximum of \\ ■ \\-norms of the columns in A. 

For proof, see Appendix B. 



4 Verifiable conditions for s-semigoodness 

We are about to demonstrate that condition SGs ,g(i^,g) from Proposition 2.1 leads to efficiently 
computable lower and upper bounds on the level of s-semigoodness. 

4.1 Verifiable sufficient conditions for s-semigoodness by Linear Programming 

Let 

Us = {u e BJ" : \\u\\i < s, \\u\\oo < 1} , 

so that Us is the convex hull of all { — 1, 0, 1} vectors with at most s nonzero entries, and for x € R", 
let be the sum of the s largest magnitudes of entries in x, or, equivalently, 



I II T 
\x\\s 1 = maxti x. 



Let 



/J-, r ^^ J [1 + ^C] maxfxj, 0] , i e P+ ^, . iin r 111 

^^^["])^ = ^ (1 + oUl, ' ^(-) = ii^^MiUi- 



Suppose S [0, 1), 9 G [l,oo) and p,cr £ [0, oo) are given. Consider the following condition on an 
m X n matrix A: 

VSG s{S,, 0, p,cr): There exist m x n matrix Y = and a vector v G such 

that 

^s{-a[Y,A]) + iA'^v), < ^, l<i<n (a) 
^smY,A])-{A'^v)^ < i^P+ (6) 
<^sia[Y,A])-{A^v), < eC,iGP+ (c) (13) 
WUiW* < a, 1 < i < n (d) 
Ikll* < P (e) 
where Ci[Y, A] is the i-th column of the matrix I — Y^A. 
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Observe that this condition is verifiable, since (13) is a system of explicit convex constraints on 
Y and v. 



Proposition 4.1 Let A satisfy VSGs(C, Pi c"") with some £ [0,1), 9 £ [1, oo), and p,a £ [0, oo). 
Then A satisfies SGs^p{^,9) with 



f3 = p + a max < 



0<k+< Card(P+) 
k+{l + 90 + kn{l + ■■ < kn < Card{Pn) }<p + as{l + 90. (14) 

k+ + kn < s 



In particular, A is s-semigood. 

For proof, see Appendix C. 

Some comments are in order. 



Origin of the condition SGs./3(^, 9). The condition VSGs(^, 9, p, a) is yielded by a simple and 
general construction, and we believe it makes sense to present this construction in its general form. 
The essence of the matter is in building a verifiable sufficient condition for the validity of (9), see 
Proposition 2.1.iv. By positive homogeneity of degree 1 of the convex functions 0, ^ participating 
in (9), the latter condition is equivalent to 

Opt := max{G(x) : Ax = 0,x G X} < ^, X = {x : ^{x) <!}. (15) 

X 

A verifiable sufficient condition for (15) is basically the same as an efficiently computable upper 
bound for Opt; the sufficient condition for the validity of (15) associated with such a bound merely 
states that the bound is < ^. Now observe that from the origin of ^ (see (9)) it is clear that X has 
a moderate number, A^, of readily available extreme points x^, ...,x^ (in the case of (9), A^ = 2n), 
so that the only difficulty in computing Opt exactly comes from linear constraints Ax = 0. The 
standard way to circumvent this difficulty and to efficiently bound Opt from above is to use the 
Lagrange relaxation: for any v £ R'", 

Opt = raayilQix) + Ax : Ax = 0,x £ X\ 

< max |e(x) + v'^Ax : x £ X] = max \@(x') + v'^Ax'], 

X ^ ^ ' ' l<i<N 

and hence the efficiently computable Lagrange relaxation bound inf„ maxi<j<7v[0(a;*) + v'^Ax^] is 
an upper bound on Opt. Unfortunately, in our situation this bound can be very poor; e.g., when 
X is symmetric with respect to the origin and G is even (as it happens in (9) when P+ = 0), it is 
immediately seen that the bound becomes the trivial bound Opt < max^^gx 0(x) = maxj 0(x*). In 
order to strengthen the relaxation, we pass to the Fenchel-type representation of 

0(x) = sup [[Pu + qfx - 0*(n)] 

u 

with a proper convex function 0*; such a representation, even with Pu + p = u, exists whenever 
is a proper convex function (and can be easily found for we are interested in). We now have for 
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any Y e R"^x", v e R™, 

Opt = max{6(x) : = 0, 2; G X} = sup{[Pn - 6*(n) : = 0, X G X} 

= sup {[Pu +p]^[x - Y'^Ax] + v^^x - e*(u) : ^x = 0,x G X} 

x,u 

< fiVLY>{[Pu + pf[x - Y'^Ax] + v^Ax - 9*(u) : x G X} 

= max sup + nl^ix* - Y'^Ax'-] + Ax" - QJu)\, 
l<i<N u 

' V ' 

■■=e^(Y,v) 

SO that the condition 

3{Y G R™^", t; G R"*) : BiiY, v) < ^, l<i<N, (16) 

is sufficient for the vahdity of (15). Note that the functions 0j, by their origin, are convex, so that 
the condition (16) is efficiently verifiable, provided that are efficiently computable. 

In the case we are interested in, the extreme points of X are the 2n vectors — Cj for 1 < i < n, 
Ci for i G Pn, and 9~^ei for i G P+, where is the i-th basic orth. Implementing the outlined 
bounding scheme and adding additional restrictions (13.d,e) to get a control over /3, we arrive at 
(13). It should be stressed that the outlined scheme can be applied to bounding from above the 
optimal value of a whatever problem of the form (15) with a convex polytope X and a proper convex 
objective 0; all what matters is that X is given as Conv{x^, x^} and is efficiently computable. 
Note also that when X is a polytope given by list of M linear inequalities, we can efficiently represent 
it as the intersection of M-dimensional standard simplex and an affine plane, so that the outlined 
scheme is applicable to a whatever problem of maximizing an efficiently computable proper convex 
function under a (finite) system of linear inequality and equality constraints. 

Effect of increasing (3,0,^. The condition SGs^p{5,,0) appearing in Proposition 2.1.v clearly is 
"monotone" in the parameters /3,0,^: whenever A satisfies this condition and > /3, 6' > 9 and 
> ^, A satisfies the condition SGs^j3'{(,' ,9') as well. Proposition 4.1 offers a verifiable sufficient 
condition for the validity of SGs,i3{S,,9), specifically, 

VSG*^^(^, 9): 3Y, v p, a satisfying (13) and the relation p + f7s(l + 9C) < /3. 

A natural question is, whether this verifiable condition possesses the same monotonicity properties 
as the "target" condition SGs^^(^,0). In the case of the affirmative answer, in order to conclude 
that A is s-semigood, we could check the validity of VSG*^^(^,0) for appropriately large values of 
/3, 9 and a close to one value of < 1; if the condition is satisfied, A is s-semigood, and error bounds 
from Proposition 3.1 take place. Were the condition VSG* f^{(,,9) "not monotone," to justify the 
s-semigoodness of A via this condition would require a problematic and time-consuming search in 
the space of parameters 13,9,(,. Fortunately, the condition VSG*^^(^,0) indeed is monotone: 

Proposition 4.2 Let A satisfy VSG* ^(^,^), and let Y,v,a,p be the corresponding certificate, that 
is, p + as{l + 9(,) < (3 and Y, v, a, p satisfy (13). Then A satisfies VSG* ^/(C', 9') whenever j3' > 13, 
9' > 9 and S^' G (C)l); ^he certificate being (Y' ,v,a, p), where the columns Y- of Y' are multiplies 
of the columns Yi of Y , namely, 

Y'-aY- [Oil 9 a- -I (1 + ^(1 + ^, ^ ^ 
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For proof, see Online Supplement F.l. 

Relation to the sufficient condition for s-goodness from [14] and the Restricted Isome- 
try Property. The verifiable sufficient condition for s-goodness from [14] requires from an mxn 
matrix A the existence of 7 < 1/2 and Y = [yi, ...,yn] £ R™^" such that 

||Ci[^,^]lki < 7, for all 1 < i < n, 

Setting 9 = 1 and ^ = (so that ^ < 1 and 7 = j^) and taking into account that in the case of 
9 = 1 we have ^s{z) < (1 + ^)||z||s,i, the latter condition implies that 

<^s{±Ci[Y,A]) < (l + 07 = e, Vi, 

that is, it implies the validity of VSGs(^, 1, 0, o"), provided that a is large enough, specifically, 
0" > WUiW* foi' all ^• 

As it was shown in the companion paper [14], when A satisfies the Restricted Isometry Property 
RIP(5, k) with parameters 5 E (0, 1), A; > 1, the above sufficient condition for s-goodness is satisfied 
with 7 = 1/3 for s as large as 0{l){\—5)^fk] as a result, a RIP((5, /c)-matrix satisfies VSGs(^, 1, 0, a) 
provided that a is large enoug h and s < 0(1)(1 - 6)Vk. Since for large m, n, m < n, typical 
random matrices possess, with overwhelming probability, property RIP(^,A;) with k as large as 
0(l)m/ ln(n/m), we see that our verifiable sufficient condition for s-semigoodness can certify the 
latter property for s as large as 0(1) y^m/ ln(n/m), provided that the matrix in question is "good 
enough" . 

4.2 Upper bounding the level of s-semigoodness 

Here we address the issue of bounding from above the maximal s = s^{A) for which A is s-semigood. 
The construction to follow is motivated by item (iv) of Proposition 2.1. A necessary and sufficient 
condition for the s-semigoodness of A is the existence of ^ < 1 and ^ > 1 such that for all x G Ker^ 
and any set / of indices with Card(/) < s 

maK[il-0x^,il + 90x,]+ {I + 0\xi\ < {x) 

ieinp+ je/np„ 

where 

^{x) = ^ max[-Xi,9xi] + ^ \xi\, (17) 
jeP+ ieP„ 

or, equivalently, 

(!) for every x G Ker^ and every vector v with at most s nonzero entries and nonzero 
entries Vi belonging to [1 — ^, 1 + ^9] if i G P+ and belonging to [— 1 — ^, 1 + ^] if i G P„, 
one has 

v^x < ^^(x). 
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Observe that the convex hull of the vectors v in question is exactly the set 

,,,g \ ^<vi<i + ei,iep+, \v^\<l + i.i^PnA 

Recalling that = {1, setting q = n — p = Card(i-'„) and 

U = {u£lC' : \\u\\i < s, ||u||oo < 1, lii > for i G P+} (18) 



we see that 

The condition (!) now reads 



(1+wp 








(1 + e)/, . 



(19) 



max V X < ^^ix) for all x G Kerj4. 
Setting X = {x & Ker^d : ^(x) < 1} the latter condition, by homogeneity reason, is the same as 

Opt = Opt(C, e) := maxiv'^x : v G Z^^'^ x £ x] < (20) 
v,x L J 

recall that A is s-semigood if and only if there exist ^ > 1 and ^ < 1 such that (20) takes place. 

We can use (20) in order to bound s*(^) from above, as follows. In order to certify that 
s^,{A) < s for a given s {s is the input to our algorithm), we fix a large 9 and a close to one ^ < 1 
(these are the parameters of the algorithm) and run the iterations 



no 



G U^'^ I— )■ xi G Argmax^g_;t'n(^x i— ui G Argmax„g^5,en"^xi i— )• .. 



initiating them by a picked at random vertex uq of U^'^. Note that the quantities ujxi, i = 1,2, ... 
clearly form a nondecreasing sequence of lower bounds on Opt. We terminate the outlined iterations 
when the progress in the bounds - the difference ufxi — uf_iXi-i - falls below a given small 
threshold, and we run this process a predetermined number of times from different randomly 
chosen starting points. As a result, we get a set of lower bounds on Opt of the form u^x, where u 
is a vertex of U^'^ and x £ X. If our goal were merely to certify that (23) is not valid for given 
s,6,^, we could terminate this process at the first step, if any, when the current lower bound u^x 
becomes > ^ (cf. [14, Section 4.1]). We, however, want to certify that s > s*(^), or, which is 
the same by Proposition 2.1.iv, that (23) fails to be true for all 6 and all ^ < 1, and not only 
for those 0,^ we have selected for our test. To overcome this difficulty, we accompany every step 
ti I— 7- x G Argma.'K^^xu'^ X by an additional computation as follows. In our process, u is an extreme 
point of U^'^, that is, a point with Su ^ s nonzero entries, let the set of indices of these entries be 
/. Setting Ci = sign(tij), we solve the following LP problem 

r Xi<0, i€ P+\I 
^ + ^ eiXi : < Ax = 



max 

X 



If the optimal value in this problem is > 1, we terminate our test and claim that A is not s-good; 
by Proposition 2.1.ii, this indeed is the case. 

As applied to a given input s, the outlined test either terminates with a valid claim "s > s^,{A)" , 
or terminates with no conclusion at all, in which case we could pass to testing a larger value of s. 
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5 Limits of performance of LP-based sufficient conditions for s- 
semigoodness 

Unfortunately, the condition in question, same as its predecessor from [14], cannot certify s-semi- 
goodness of an m x n matrix in the case of s > 0{l)^/rn, unless the matrix is "nearly square". The 
precise statement is as follows (cf. [14, Proposition 4.2]): 

Proposition 5.1 Let 

n > 2(2\/2^+ 1)^ (21) 

and let^<l,9>l,a>0,p>0, an integer s and an m x n matrix A be such that A satisfies 
YSGs{^.,9,p,a). Then 

s<2\/2^+l. (22) 

For proof, see Appendix D. 

The results from Proposition 5.1 show that our verifiable sufficient conditions can only certify 
s-semigoodness of an m x n matrix at a suboptimal rate of s < 0{l)y/m, unless the matrix is 
"nearly square". In fact this verifiable bound can still give a very poor impression on the true 
largest s = s^{A) for which A is s-semigood. An instructive example in this direction is as follows. 
Consider the case of -P+ = {1, n}, let m = 2d + l be odd, and let the rows of A be comprised of 
the values of basic trigonometric polynomials 

Po{(l)) = l, p2i-i (</>) = cos(z(^), p2i{(l)) = sm{i<j)), l<i<d, 

taken along the regular grid cpj = 2TTj/n, < j < n, so that Aij = Pi{(pj), 0<i<m, 0<j<n (we 
enumerate rows and columns starting with rather than with 1). It is well known [5, 10] that in 
this case A is s-semigood for s = d. In contrast to this, when A is not "nearly square" , specifically, 
when n > ind, A can satisfy the condition \'SGs{(,,9, p,a) only for s < 2, no matter how large 
6,a,p are and how close to 1 ^ < 1 is, see Online Supplement F.2. 

6 Verifiable sufficient conditions for s-semigoodness by Semidefi- 
nite Relaxation 

Following d'Aspremont and El Ghaoui [7], we are about to derive another verifiable sufficient 
condition for s-semigoodness, now - via semidefinite relaxation. The construction to follow is 
motivated by the development in the beginning of Section 4.2, according to which s-semigoodness 
of A is implied by the validity of (20) for > 1 and ^ < 1. 
Let, as before, 

X = {x£ KevA : "^{x) < 1} and U^'^ = {V^'^^u : u£U}, 

where ^, U and V^'^ are defined in, respectively, (17), (18) and (19). The condition (20) is 
equivalent to 

max^{V^'^ufx : ueU, x e <C- (23) 
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Observe that for u x £ X the matrices U 
3t £W,V £ S2",A G : 



uu 



P = ux"^ and X = xx'^ satisfy the relations 



(a) 



u 


p ' 




X 










< : 



>=0; 





yl2 - 


^ < 


, ' 



=v 



(c) X 



[ -Ip 





1/ 


" 










I, \ 



'^y, Tr{V) < s, 
Vi,j 



(24) 



^^'^iJ + EiGpJ^.jl ViG P+, 

(c?3) < 1 Vi, < s; 

(e) AXA^ = 0. 



Besides this, 



X = Tr(y^'^P^). 

Indeed, the latter relation, same as (24. a) and (24. e), is evident. To verify (24.6), let 
= max[ii, 0], U- = max[— 0], where max is acting coordinate- wise. Then 



U 



L 
L 



T 


T 


U-u\. 


U-uL 



L 



T 


T ' 


u+uL 









^[n_u:^ + U-\.uL] 





and the matrix V we have just defined clearly satisfies all requirements from (24.6). To 
verify (24. c), observe that the extreme points of the set Af^ = {x : ^{x) < 1} D X are 
the vectors ztcj, i > p, and — Cj, 9~^ei, i < p, so that x = FX with A G R+", Yli — 1; 
setting A = AA-^, we satisfy (24. c). To satisfy (24. d), it suffices to set U = \ui\ for all i 
and to take into account that max[—Pij,9Pij] > \Pij\ for all i,j due to > 1, and that 
Ui>0 for i £ P+. 

It follows that a sufficient condition for (23) is 

Opt := max ^ ^Tr{V^'^P'^) : (24) is satisfied} < 



max 

x,u e S", y.A e s^", 
p g R">^", t e R" 



(25) 



The optimization problem in (25) clearly reduces to a semidefinite maximization program S; by 
weak duality, the optimal value in the semidefinite dual P to 5 is > Opt. It follows that the 
efficiently verifiable condition 

Opt(P) < ^ 

is a sufficient condition for s-semigoodness of A. Note that the above construction depends on 
9 > 1 and < 1 as parameters. 
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Remark. Consider the case of P+ = 0, where X = {x ^ : ||x||i < 1, Ax = 0} D Z = {x R" : 
< 1}. In this case, the standard semidefinite relaxation of the set C^, = Conv{xx'^ : x G Z} is 



X 



X h 0,^ < 1 



(cf. [/]). Note that (24. c) uses another semidefinite relaxation of C*, namely, 



X : 3A G S 



2n 



A^O,Ai,j- >0 Vi,i, Ei, Aij<l 



^ — [In, —In]^[In, —In_ 



It is immediately seen that C C C C; a surprising fact is that the second of these inclusions is 
strict. Thus, the relaxation of given by C is less conservative than the standard relaxation given 
by C. As observed by A. d'Aspremont (private communication), the relaxation C can be further 
improved, namely, by replacing C with 



C~ 



X : 3A 



^11 ^12 
^21 ^22 



G S 



2n 



^i,u g j^nxn^ ^ ^ 0, Aij > Vi, j 

X = \L,-UA\Ir,,-Ir 



l<i<n 

IT 



Note that this idea can be used to improve the semidefinite relaxation given by C as well. Specifi- 
cally, the matrix V as built in the justification of (24) clearly satisfies {V^'^)ii = 0, 1 < i < n, and 
we can add these linear constraints on V to (24.&). Similarly, when representing a vector x G 
as FX with A G R4", Aj < 1, see the justification of (24), we clearly can ensure that XiXn+i = 0, 
1 < i < n, that is, the matrix A we have built in fact satisfies Ai^n+i = A„+j^j = 0, 1 < i < n, and 
we can add these linear constraints on A to (24. c). 



7 Numerical results 

In order to compare the performance of the proposed bounds on the maximal s = s^,(A) for which 
a given matrix. A, is s-semigood, with the bounds known from the literature, we present some 
preliminary numerical results for relatively small sensing matrices. Our goal is to see if the sign 
information on a signal allows to improve the bounds for s*(^) as compared to the bounds on the 
largest s = so{A) for which A is s-good. 

We generate four sets of random matrices, which are normalizations (all columns scaled to be 
of II • ||2-norm 1) of (a) Rademacher matrices (i.i.d. entries taking values ±1 with probabilities 
0.5), (b) Gaussian matrices (iid AA(0, 1) entries), (c) Fourier matrices — m x n submatrices of the 
matrix of n x n Discrete Fourier Transform, and (d) Hadamard matrices — m x n submatrices of 
the nxn Hadamard matrix^; in the cases (c,d), the m rows comprising the submatrix were drawn 
at random from the n rows of the "parent" matrix. For each type, we set the number of columns 
to n = 256 and vary the number of rows, m = 0.5n, . . . , 0.95n. 

We bound from below the value sq{A) using the bound by mutual incoherence and the 
bounds s[ai] and s[as], computed through the LP-based verifiable sufficient conditions for s- 
goodness (see [14, Section 6]). 

■^The Hadamard matrix Hd, d = 0, 1,2, has order 2^* x 2^* and is given by the recurrence Ho — 1, Hd+i = 
[Hd, Hd; Hd, —Hd]. 
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The lower bound on s*(j4) is computed by invoking condition VSGs(^, 9, p, a), where p = a = oo 
and 9 is set to once for ever fixed "large enough" value, and ^ is set to 0.9999, see section 4.1 and 
Propositions 4.1, 4.2. Note that given a matrix Y, and setting t; = 0, one can compute the largest s 
satisfying (13) and thus ensuring the validity of YSGs{£,,9, p,a). We first compute the best lower 
bound s on s^:{A) given by the Y-matrices generated when bounding sq{A). Then we compute 
the "improved" lower bound for as follows: we check whether the condition VSGs 0, p, cr) 

holds true for s = s + 1, if it is the case, check whether this condition holds true for s = s + 2, and 
so on. 

While the outlined lower bounds on s*(j4) and sq{A) are efficiently computable via LP (when 
a = p = oo, the sufficient condition is easily checked by solving a Linear Programming program), the 
sizes of the resulting LPs are rather large. For instance, when A is mxn, the LP associated with (13) 
has a (2n^+2n+l) x ((m+2n)(n+l)+2) constraint matrix (compared to (2n^+n) x (n(m+n+l) + l) 
constraint matrices arising when computing lower bounds for sq{A)). For instance, for m = 230 and 
n = 256, bounding s^{A) results in an LP program of the size 131, 585 x 190, 696, while computing 
a lower bound on sq{A) requires solving an LP problem of size 131,328 x 124,673. In all the 
computations, we used the state-of-the-art commercial LP solver mosekopt [1]. 

The upper bounds on s^:{A) and on sq{A) are computed by the techniques from Section 4.2 and 
[14, Section 4.1]. 

The results of our experiments and related CPU times are presented in Table 1. The com- 
putations were carried out on a single core of an 8-core Intel Xeon E5520@2.27GHz CPU Linux 
workstation. 

The results in Table 1 merit some comments. We observe that our LP-based efficiently com- 
putable lower bounds on so{A) and s*(^) clearly outperform the bounds based on mutual incoher- 
ence. We notice that for Fourier and Hadamard matrices, the lower bounds on s*(A) and so{A) 
are nearly always the same, except for two Hadamard instances with m = 230 and m = 242. On 
the other hand, for Gaussian and Rademacher matrices, as the number of rows m approaches the 
number of columns n, the difference between the best certified lower bounds on s^,(A) and on sq{A) 
increases (for the sizes we have considered, this difference attains 5 for the Gaussian matrix with 
m = 242). While for Gaussian, Rademacher and Fourier matrices, the upper bounds on s^{A) 
become loose (they are twice or three times higher than the upper bounds on so{A)), these bounds 
become tighter in the case of Hadamard matrices. Further, for some matrices the lower and the up- 
per bound on S(){A) match (e.g., the Hadamard matrix with m = 152), what allows to identify the 
exact value of sq{A) . Moreover, we have observed samples of smaller random Hadamard matrices 
(with n = 128) for which the lower bounds and upper bounds on both s*(^) and sq{A) coincide, 
which implies s^,(A) = so{A) in these cases. 

8 Matching pursuit algorithm 

The Matching Pursuit algorithm for signal recovery has been first introduced in [15] and is motivated 
by the desire to provide a reduced complexity alternative to the -recovery problem. Several 
implementations of Matching Pursuit has been proposed in the Compressive Sensing literature 
(see, e.g., the review [2]). All of them are based on successive Euclidean projections of the signal 
and the corresponding performance results rely upon the bounds on mutual incoherence p{A) of 
the sensing matrix. We are about to show that the LP-based verifiable sufficient conditions from 
the previous section can be used to construct a specific version of the Matching Pursuit algorithm 
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Table 1: Comparison of efficiently computable bounds on n = 256 



Fourier matrices 





Unsigned 


Nonnegat ive 


CPU time (s) 




LBs on 




TTR 

U D 


LB 


TTR 


Unsigned 


Nonncgativc 




s[fi] 


s[qi 




g 




g 


s[qi] 


s[as] 


s 


s[as] 


s 


128 


3 


5 


5 


12 


5 


47 


0.8 


1054.0 


146.0 


3114.4 


172.9 


128 


3 


5 


5 


11 


5 


32 


0.9 


986.0 


169.4 


2891.5 


311.5 


152 


2 


6 


6 


11 


6 


49 


1.1 


898.5 


252.5 


3680.2 


179.6 


152 


3 


6 


6 


11 


6 


53 


1.3 


899.3 


161.7 


3836.7 


183.5 


178 


2 


6 


6 


12 


6 


47 


1.1 


866.5 


228.6 


3976.0 


294.0 


178 


3 


7 


7 


16 


7 


42 


0.7 


484.8 


365.2 


3216.8 


416.9 


204 


4 


8 


8 


17 


8 


67 


1.0 


828.5 


235.4 


3829.7 


209.2 


204 


3 


7 


7 


15 


7 


65 


1.1 


906.8 


220.2 


3914.4 


197.4 


230 


4 


10 


10 


21 


10 


70 


1.1 


1879.9 


300.5 


4287.6 


384.6 


230 


4 


9 


9 


20 


9 


65 


1.0 


856.6 


286.5 


4040.2 


362.0 


242 


5 


11 


11 


26 


11 


89 


1.7 


1425.1 


290.5 


6444.1 


513.0 


242 


4 


10 


10 


19 


10 


75 


1.2 


1920.6 


265.3 


4069.1 


232.8 



Hadamard matrices 





Unsigned 


Nonnegat ive 


CPU time (s) 




LBs on 


so (A) 


UB 


LB 


UB 


Unsigned 


Nonncgative 


m 


s[ti] 




] 4cs] 


s 


s[qs] 


s 


s[qi] 


s[as] 


s 


s[as] 


s 


128 


3 


5 


5 


7 


5 


8 


0.2 


1148.1 


77.8 


3007.0 


68.5 


128 


2 


5 


5 


7 


5 


7 


0.3 


1297.1 


73.4 


2894.4 


116.8 


152 


3 


7 


7 


7 


7 


58 


0.3 


1224.4 


47.9 


3997.0 


186.8 


152 


4 


7 


7 


13 


7 


58 


0.2 


1205.8 


245.0 


3962.6 


310.4 


178 


4 


9 


9 


15 


9 


70 


0.2 


1269.8 


238.9 


4828.2 


212.0 


178 


4 


9 


9 


15 


9 


19 


0.3 


1340.7 


271.1 


4923.3 


342.8 


204 


4 


12 


12 


15 


12 


16 


0.5 


2908.1 


131.2 


6409.9 


385.4 


204 


5 


12 


12 


15 


12 


16 


0.4 


2996.7 


148.9 


5507.9 


253.9 


230 


8 


18 


18 


31 


19 


31 


0.3 


1860.1 


250.8 


9046.7 


331.1 


230 


8 


18 


18 


31 


18 


39 


0.4 


2100.2 


282.8 


4081.3 


396.8 


242 


12 


26 


26 


31 


27 


31 


0.3 


2015.1 


92.7 


7478.2 


176.2 


242 


12 


26 


26 


31 


26 


31 


0.3 


1976.7 


116.8 


3597.9 


412.0 



Rademacher matrices 





Unsigned 


Nonnegat ive 


CPU time (s) 




LBs on 


so{A) 


UB 


LB 


UB 


Unsigned 


Nonncgative 


m 


s[ti] 


s[ai 


] s[as] 


s 




s 




s[as] 


s 




s 


128 


1 


5 


5 


14 


5 


53 


27.8 


1253.1 


171.6 


3388.7 


124.8 


128 


1 


5 


5 


15 


5 


48 


27.8 


1361.5 


191.1 


3291.6 


123.4 


152 


2 


6 


6 


18 


7 


65 


38.4 


1426.3 


322.7 


9592.1 


136.3 


152 


1 


6 


6 


19 


7 


66 


38.3 


1183.0 


218.9 


9146.3 


139.0 


178 


2 


7 


8 


25 


9 


78 


44.2 


2819.1 


258.9 


8032.1 


225.8 


178 


2 


7 


8 


24 


9 


78 


41.8 


2481.7 


256.0 


8306.3 


168.2 


204 


2 


10 


11 


32 


12 


92 


51.1 


1434.2 


291.8 


9738.5 


209.3 


204 


2 


10 


11 


30 


12 


90 


50.8 


1316.6 


448.3 


9146.8 


345.4 


230 


2 


14 


16 


41 


19 


107 


61.8 


2422.9 


302.7 


15235.2 


162.2 


230 


2 


14 


16 


39 


19 


107 


61.7 


2466.2 


624.0 


15578.4 


161.9 


242 


2 


20 


23 


47 


27 


116 


64.8 


3929.4 


269.2 


19828.7 


178.1 


242 


2 


19 


23 


47 


27 


111 


68.0 


4242.4 


277.8 


20506.7 


270.5 



Gaussian matrices 





Unsigned 


Nonnegat ive 


CPU time (s) 




LBs on 


so{A) 


UB 


LB 


UB 


Unsigned 


Nonncgative 


m 


s[fi] 


s[ai 


] slas] 


s 


s[Os] 


s 




s[as] 


s 


s[as] 


s 


128 


1 


5 


5 


14 


5 


44 


28.2 


852.1 


172.4 


3283.2 


114.7 


128 


1 


4 


5 


15 


5 


52 


27.7 


1913.9 


177.7 


3712.0 


124.6 


152 


2 


6 


6 


19 


7 


58 


35.4 


981.0 


214.1 


8433.5 


392.8 


152 


1 


6 


6 


19 


7 


58 


38.9 


1004.0 


242.6 


8231.7 


373.3 


178 


2 


7 


8 


24 


9 


79 


43.0 


2164.4 


393.9 


10294.7 


368.2 


178 


2 


7 


8 


25 


9 


77 


47.6 


2390.3 


263.1 


9548.8 


374.0 


204 


2 


10 


11 


32 


12 


88 


58.0 


1363.6 


293.3 


11496.7 


274.1 


204 


2 


10 


11 


32 


12 


915 


51.7 


1218.4 


293.4 


12497.2 


529.5 


230 


2 


14 


17 


41 


19 


102 


70.4 


3200.9 


339.7 


18771.3 


431.6 


230 


2 


14 


16 


39 


19 


106 


61.5 


2118.4 


485.4 


18959.5 


435.0 


242 


2 


19 


22 


46 


27 


113 


73.6 


2212.8 


277.4 


26874.6 


269.2 


242 


2 


20 


23 


47 


27 


112 


65.3 


2995.2 


426.7 


21308.7 


191.7 



which we refer to as Non-Euclidean Matching Pursuit (NEMP) algorithm. 

Suppose that we have in our disposal r, r± > and a matrix F = [yi, such that 

(a) -r_ < [/ - y^^],,- < r+, Vi G P+, Vj, 

(6) -r < [/ - y^A],j < r, Vi G P„, Vj, (26) 
(c) ||yj||*<o-, Vj. 

Consider a signal w; G "Pn such that — < ^, where tt;* is the vector obtained from w 
by replacing all but s largest magnitudes of entries in w with zeros, and let y and 5 be such that 
\\Aw — y\\ < 6. 

Suppose that 

/9 = s max{T+, r„, r} < 1. (27) 
To simplify notation, we denote max[a, b] by a V b. Consider the following iterative procedure: 

Algorithm 1 

1. Initialization: Set = 0, = . 

2. Step k, k = 1,2, Given v^''"^^ G R" and ak-i > 0, compute 



(a) u = Y'^{y — Av^^ and n segments 

S^ = 

Define A G R" by setting 



Hi - r_afc_i - a5, Ui + T+afc_i + cf5], i G P+, 
Ui - TOk-i - a6, Ui + TQfc„i + cr6], i G P„. 



[ui - T-Uk-i - cfS\ + , i e P+, 
Ai= { [ui - rafc_i - a6]+, i G P„, Ui > 0, 
-[\ui\ - TUk-i - a6] + , i G P„, Ui < 

(here [a]+ = max[0, a]J. 
(b) Setv^^^ =t;('=-i) +A and 

Uk = s[2t V (r_ + r+)]afc_i + 2scr(5 + /x. (28) 

and /oop to step k + \. 
3. The approximate solution found after k iterations is v^^\ 

Proposition 8.1 Assume that > for i G P+, (27) takes place, and that \\w — w'^\\i < ji with 
a known in advance value of ^. Then the approximate solution v^^^ and the value Uk after the k-th 
step of Algorithm 1 satisfy 



(ofc) for all i w^-'^'' G Conv{0; Wj}, (6fc) — u'-'^-* ||i < 



oik- 
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For proof, see Appendix E. 
Let 

A = s[2rV(r_+r+)]; 

if A < 1, then also p < 1, so that Proposition 8.1 holds true. Furthermore, by (28) the sequence ak 
converges exponentially fast to the limit Uoo ^ A ^ • 

Ok = X''[ao - aoo] + "oo- 

Note that when P+ = 0, we can set t_ = t+ = to obtain A = 2st; in the case of P^i = by 
setting r = 0, we have A = s(r_ + t_|_). 

The bottom line is: if the optimal value in the convex program 



Opt = min < 

r,T±,Y ^ 



-r_ < [/ - Y^Alij < T+, Vi G P+, Vi 
s[2t V (r_ + r+)] : -r < [/ - Y^A],j < r, Vi G P„, Vj 

r, T± > 



is < 1, the above procedure, as yielded by an optimal solution to the latter problem, possesses the 
following properties: 

1. All approximations v^^\ A; = 0, 1, ... of w are supported on the support of w; 

(k) 

2. For i G P+, > are nondecreasing in k and are < Wi for all k; 

3. For i G P„, 

(k) (k) 

• Wi > 0, then < ^ Wi and are nondecreasing in k; 

• a Wi < 0, then < v^^^ < and v^^^ are nonincreasing in /e; 

4. As /c grows, the upper bound ak on the £i-error of approximating w by v^^^ goes exponentially 
fast to 

2sa6 + ^ 
1-Opt- 

Let now ^g[0,1),(T>0 and 6* > 1 and suppose that an m x n matrix A satisfies the following 
condition: 



VSGs(^, cj, 6): There exists m x n matrix Y = [yi, ...,yn] such that < a for aU i 

and 

- (T^i < - Y^A],, < « V. P+, Vj, 

-(t4^<[/-1^^^W< (t4^ V.GP+, Vj0P+, (29) 



Observe that (29) is a system of convex inequalities in Y. Further, VSGs(^, a, 9) certainly implies 
VSGs(,^, 0, 0, <t), and is therefore sufficient condition for s-semigoodness of the matrix A. 
When VSGs(C,cr,6') is satisfied with ^ G (0, 1) and 61 > 1, by taking 

(1 + ^0)^' ^+ (i + ee)s ^ (i + e)s' 
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we obtain 

Combining this condition with Proposition 8.1 gives: 



Corollary 8.1 Suppose that A satisfies the condition VSGs(^, o", 0) with certain ^ G (0, 1), o" > 
and 6 > 1. Let w G Vn be a vector with — < fj, where is the vector obtained from w by 
replacing all but s largest in magnitude entries in w with zeros, and let y be such that \\Aw — y\\ < 6. 
Then the approximate solution found by Algorithm 1 after t iterations satisfies vf^ > for all 
i G P+ and 



,^^ 2sa5 + /i j 

\w - v^^'Wi < + 

1 — A 

id 



Y'^y\\s,i + sa6 + fi 2sa6 + n 



1- p 1 - A 



where A is given by (30) and p — . 

It should be noted the NEMP algorithm has several drawbacks as compared with the £i-recovery. 
First, the pursuit algorithm requires a priori knowledge of several parameters (a, Y, r, r_, r+, s 
and p). Second, the value (1 — X)~^{2sa5 + p) is a conservative upper bound on the error of the ii- 
recovery, but the error bound in Corollary 8.1 is exact. On the other hand, the NEMP algorithm can 
be an interesting option if the £i-recovery is to be used repeatedly on the observations obtained with 
the same sensing matrix A; the numerical complexity of the pursuit algorithm for a given matrix A 
may only be a fraction of that of the ^i-recovery, especially when used on high-dimensional data. 

Our concluding remark is on the condition 

(31) 



l + p{A) 2s' 

where p{A) is the mutual incoherence of A (see (6)). This condition is usually used in order to 
establish convergence results for the Matching Pursuit algorithms (see, e.g. [12, 13, 3]). As it 
is immediately seen, when p{A) is well defined (i.e., all columns in A are nonzero), the matrix 
Y = [yi, ...,yn] with the columns 

(1 + p{A))AjA, 
satisfies for all i = 1, m and j = 1, n the relations 

p{A) 



\[I-Y'AU< 



l + p{Ay 



In the case of (31), setting 6 = 1 and specifying ^ from the relation = ; we get < 

^ < 1 and meet all inequalities in (29). It follows that Y certifies the validity of the condition 
VSGs(.^,(T, 1) with the outlined ^ and with all a > max , . ,,2 , and thus the above Y can 

be readily used in Matching Pursuit. Note that in the situation in question Corollary 8.1 recovers 
some results from [12, 13, 3]. 



18 



References 

[1] Andersen, E. D., Andersen, K. D., The MOSEK optimization tools manual. 
http: / /www. mosek.com/fileadmin / products /6_0 /tools /doc /pdf /tools. pdf. 

[2] Bruckstein, A., Donoho, D., Elad, M., From Sparse Solutions of Systems of Equations to 
Sparse Modeling of Signals and Images, to appear in SI AM Review (2009). 

[3] Bruckstein, A., Elad, M., Zibulevsky, M., A non-negative and sparse enough solution of an 
underdetermined linear system of equations is unique, IEEE Transactions on Information 
Theory, 54(11), 4813-4820 (2008). 

[4] Candes, E.J., Compressive sampling, Marta Sanz-Sole, Javier Soria, Juan Luis Varona, Joan 
Verdera, Eds. International Congress of Mathematicians, Madrid 2006, Vol. Ill, 1437-1452. 
European Mathematical Society Publishing House, (2006). 

[5] Caratheodory, C, Ueber den variabilitaetsbereich der fourierschen konstanten von positiven 
harmonischen funktionen. Rend. Circ. Mat. Palermo, 32, 193-217 (1911). 

[6] Cohen, A., Dahmen, W., DeVore, R., Compressed sensing and best k-term approximation, 
http://www.math.sc.edu/devore/publications/CDDSensing 6.pdf, submitted for publi- 
cation (2006). 

[7] d'Aspremont, A., El Ghaoui, L., Testing the Nullspace Property using Semidefinite Program- 
ming, http://arxiv.org/abs/0807.3520, submitted to Math. Programm. (2008). 

[8] Donoho, D., Elad, M., Optimally sparse representation in general (nonorthogonal) dictionaries 
via ^1 minimization Proc. of the National Academy of Sciences, 100(5), 2197-2202, (2003). 

[9] Donoho, D., Huo, X., Uncertainty principles and ideal atomic decomposition IEEE Transac- 
tions on Information Theory 47(7), 2845-2862 (2001). 

[10] Donoho, D., Tanner, J., Sparse Nonnegative Solutions of Underdetermined Linear Equations 
by Linear Programming Proc. of the National Academy of Sciences, 102(27), 9446-9451, 
(2005). 

[11] Donoho, D., Tanner, J., Neighborliness of randomly-projected simplices in high dimensions 
Proc. of the National Academy of Sciences, 102(27), 9452-9457, (2005) 

[12] Donoho, D., Elad, M., Temlyakov, V., On Lebesgue-Type Inequalities for Greedy Approxi- 
mation Journal of Approximation Theory, 147(2), 185-195 (2007). 

[13] Elad, M., Optimized projections for compressed sensing IEEE Trans, on Signal Processing, 
55(12), 5695-5702 (2007). 

[14] Juditsky, A., Nemirovski, A., On Verifiable Sufficient Conditions for Sparse Signal Recov- 
ery via Minimization http : / /hal . archives-ouvertes . f r/hal-00321775/ , submitted to 
Math. Programm. (2008). 

[15] Mallat, S., Zhang, A., Matching pursuits with time-frequency dictionaries IEEE Transactions 
on Signal Processing, 41(12), 3397-3415 (1993). 



19 



[16] Zhang, Y., A simple proof for recoverability of ell- 1-minimization: go over or under? Technical 
Report TR05-09, Department of Computational and Applied Mathematics, Rice University, 
Houston, TX (2005). 

[17] Zhang, Y., A simple proof for recoverability of ell- 1-minimization (II): the nonnegative case. 
Technical report TR05-10, Department of Computational and Applied Mathematics, Rice 
University, Houston, TX (2005). 

[18] Zhang, Y., Theory of Compressive Sensing via 11 minimization: a non-rip analysis and ex- 
tensions. Technical report TR08-11, Department of Computational and Applied Mathematics, 
Rice University, Houston, TX (2008). 



A Proof of Proposition 2.1 

(i)=^(ii): Let A be s-semigood, and let, in contrast to what is stated by (ii), J be a subset of 
{1, ...,n} with Card(J) < s and x E KerA\{0} be such that Xj < for all i G P+ \ J and 

ieJnP+ ieJnPn i^J 

Let / = (Jn Pn) U {i G Jn P+ : Xj > 0} so that I J. From the construction of /, we have Xj < 
for i £ J \ I implying that Xj < for i £ \ I. Further, 

^ ^ ~\~ ^ ^ |Xj| — ^ ^ Xj ^ ^ Xj -|- ^ ^ |Xj| 

ie/nP+ ieinPn i&JnP+ i&.J\i ieJnP„ 

— ^ ^ ^ ^ 27 j — ^ ^ |Xj| -|- ^ ^ |Xj| — ^ ^ |Xj|. 

Hence / also violates the condition in (ii). Setting Ui = Xj when i £ I and Uj = otherwise and 
setting V = u — X, we have Ui > for any i £ I Ci P+, Uj = for any i G P+ \ /, and > for 
i G P4- \ /, "yj = for i £ I D P+ and \ui\ > In addition, Au = Av due to Ax = 0, and u 

is s-sparse; finally, u ^ v due to x / 0. We see that the s-sparse vector u G "Pn is not the unique 
solution to 



mm 



1^ |zi| : Az = Au, > Vi G 



which is a desired contradiction. 

(ii)=^(iii): Let A satisfy (ii). Let J be the family of all subsets J of {1, ...,n} of cardinality 
< s. For J £j, let 

Xj = {x£ KexA : ||x||i = 1, Xj < Vi G P+ \ J}. 
Assuming that Xj / 0, let x G Xj. By (ii), we have 



We claim that X^j^j |xj| > 0. 
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Indeed, otherwise Xj 7^ implies that i £ J. Let /+ and /_ be the subsets of J such 
that > for i £ and Xi < for i G I^. At least one of these sets is nonempty 
due to X 7^ 0. W.l.o.g. we can assume that — Yli^i^ (otherwise we could 

replace x with —x and swap /+ and /_). Applying (ii) to x and to 1+ in the role of J, 
we should have 

je/+nP+ ie/+nP„ 

which is not the case. This contradiction shows that X^^^j \xi\ > whenever x E Xj. 
From our claim it follows that the function 

is continuous on Xj and is < 1 at every point of this set. Since Xj is compact, we conclude that 
when J £ J is such that Xj ^ 0, there exists < 1 such that 

^ Xi + ^ |xi| < O X] for any x G Xj. 

Setting ^ = max ^j, we clearly ensure the validity of (iii). The implication (ii)=^(iii) is proved. 

(iii)=^(i): Let (iii) take place; let us prove that A is s-semigood. Thus, let u with Uj > for 
all i E P+ be s-sparse; we should prove that u is the unique optimal solution to the problem 



mm 



1^ : Az = Au, > V? E 



Assume, on the contrary to what should be proved, that the latter problem has an optimal solution 
V different from u, and let x = n — t;, so that x E KerA and x 7^ 0. Setting / = {i : Uj 7^ 0}, we 
have Card(/) < s and Xj < when i E P+ \ /, whence by (iii) 

^ Xi+ ^ \Xi\ < ^^\Xi\ = ^^\Vi\, 

ieir\P+ i&inP„ i^i i^i 

whence also 

^ ni+ ^ \ui\ < \vi\+C^\vi\- (32) 

ie/np+ ieinPn ie/nP+ je/nP„ i^i 

Since < = X^jg/ \ui\ due to the origin of v, (32) implies that = 0) that is, 

both u and are supported on /, so that x is supported on / as well. Now let /+ = E / H P+ : 
Xi > 0}, /„ = {z E / n P+ : Xj < 0} and /„ = / Pi P„. Replacing, if necessary, x with — x and 
swapping /+ and /_, we can assume that — Sje/+ l^' l — Sie/- ^PPly™S (iii) to x 

and to U /„ in the role of J, we get 

^ Xi + ^ |Xi| < C ^ kil. 
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thereby Eie/+ ^» = SiG/„ 1^*1 = Eie/- 1^*1 = ^ '^^^ *° Sie/+ ^» - Eie/- 1^*1- Thus, x = 0, which 
is a desired contradiction. 

We have proved that the properties (i) ~ (iii) of A are equivalent to each other. 

(iii) <;4>(iv): The imphcation (iv)=^>(iii) is evident. Let us prove the inverse implication. Thus, 
let A satisfy (iii) (and thus - (i) - (ii) as well), and let ^' G (^, 1). Let, as above, J' be the family 
of all subsets J of {1, ...,n} of cardinality < s. Let X = {x ^ Ker^ : ||x||i = 1}, and let J ^ J . 
Let X € X. We claim that there exists a neighborhood Ux of x in X and 9j^x G [IjOo) such that 
for any u G Ux and 6 > Oj^x it holds 

Y Ui+ Y \ui\<£,'l ^ \ui\+ ^ ma-x[-Ui,9ui]\ . (33) 
ieJnP+ ieJnPn \iePn\J ieP+\J J 

The claim is clearly true when there exists i E P+ \ J such that Xj > 0. Now assume 
that Xj < for i € P+ \ J. Then Xli^j \ > 0- Indeed, otherwise Xj = for all 
i ^ J, which combines with s-semigoodness of A and the relation Ax = to imply 
that X = (since assuming x 7^ 0, we have x = u — v with s-sparse it > 0, u > with 
non-overlapping supports, and Au = Av due to Ax = 0, which of course contradicts 
the s-semigoodness of A), while x definitely is nonzero (since ||x||i = 1 due to x G X). 
Now, since x G Ker^ and x,, < 0, i € P+ \ J, we have 

^ Xi+ ^ \xi\<iY\xi\ <i'Y\xi\ 

where the first inequality is due to (iii), and the second - due to Yli^j 1^*1 > 0- The 
concluding strict inequality clearly implies the validity of (33) with 9 = 1, provided that 
Ux is a small enough neighborhood of x. Thus, our claim is true. 

From the validity of our claim, extracting from the covering {Ux}xgx of the compact set X a finite 
subcovering, we conclude that there exists 9j G [l,oo) such that 

\/{x G X, 9 > 9j) : ^ X,; + ^ l^il < ^' ^ \xi\+ ^ max[-Xi,9xi] 
ieJnP+ ieJnPn \ieP„\J ieP+\J 

Setting 9 = maxjgj 9j, we see that A satisfies SGs(.^', 9). 

(iv) =^(v): Let A satisfy SGs{(,,9) for certain ^ G (0,1), 9 G [l,oo) and let || • || be a norm on 
R™. Let, further, P be the orthogonal projector of R"' on Ker^. Then clearly with a properly 
chosen C one has 

\\Px-x\\i < C\\Ax\\ 

for any x G R". Now let J be a subset of {1, ...,n} of cardinality < s, x G R" and u = Px. We 
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have 



X] + X] < Ui+ \ui\ +y^jui- Xi\ 

ieJnP+ ieJnPn i&JnP+ ieJnP„ i&j 



< C 



< e 



< e 



< e 



jeP„\J i&P+\J 



Hi Xi 



jeP„\J jeP+\J 



Ui Xi 



+ max[— Xj,6'xi 

jeP„\J jeP+\J 

jeP„\J i&P+\J 



+ max[l, 0^] ||x — u||i 
+ max[l,e^]CPx||, 



so that A satisfies SGs^^(^, 0) with /3 = max(l, 0,^)C. The impHcation (iv)=^>(v) is proved. 
(v)=^(vi)=^(iii): These impUcations are evident. □ 



B Proof of Proposition 3.1 

Let / be the support of w^, I be the complement of / in {1, ■■■,n}, and let z = u; — x. We denote 
I-f- = {i £ I : Zi > 0} , = {i £ I : Zi > 0} , and /_ = / \ /+, /_ = / \ Zf . Observe that w is a 
feasible solution to (10), so that 

\\x\\i < \\w\\i + V. (34) 

Obviously, \xi\ — \wi\ > —\zi\ and \xi\ — \wi\ > \zi\ — 2\wi\. Now using Xi,Wi > V« G P+, and 
> Vz G /+, we get 



XA - \Wi 



[by (34)] 



iG/+nP+ 



iePn 



> 



j6/+nP+ j6/-nP+ iG/-nP+ jG/+nP+ 
je/nP„ ie/nP„ 



or, equivalently. 



Yl,iei-r\P+ 1^*1 + X]je/-nP+ 1^*1 + Z]je/nP„ 



(35) 
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On the other hand, we have 

ll^zll = \\Aw - Ax\\ < \\Aw - y\\ + \\Ax - y\\ < e + 5. 
Then by condition SGs^i3{C, 0) with (/+ n P+) U (/ n P„) in the role of J, we get 

Zi+ Yl Nil ^/5P^II+^[Sie/npJ^i| + Eie(JnP+)u(/_nP+)V'(2i) 
iei+nP+ ieinPn 



(36) 



K < f3\\Az\\+C 



Zi\+Y. . _ \Zi 



\zi\ + - Zi 



^iei+nP+ 



(37) 



=r(e) 



Let us derive a bound on t(^). Now (35) imphes, independently of whether SGs^piC, 0) is or is not 
true, the first inequality in the following chain: 

T{e) < V+ Y Y 1-^*1+ Y + Y Y 

iei+nP+ ieinPn ie/+nP+ je/nP„ jeJ+nP+ 

Wil [since Wi > Zi for i G P+] 

iG/+nP+ ieinPn 

< u + K + {1 + 9)fi, [since ^ > 1 and X^jgj Itfj] < /i], (38) 



and, in particular. 



t(1) = ^ + \zi\ < v + K + 2fj.. 
ie/_nP+ ig/ 



(39) 



Combining (36), (37) and (38), we obtain 

K<p{e + S)+^[u + K + {l + e)fj] 

and thereby. 



K= Y Y 1^*1 - 

iei+nP+ ieinPn 



l3{e + 5) + ^{i^ + {9 + l)lj,) 



1-e 



Summing up the latter inequality and (39), we obtain 



i^iii = Y 1^*1+ Y 

ieinPn i£i+nP+ 



Y 1^*1 + 2^1^* 



1-e i-e ' 1-^ 



which is (11). 

To show (12) observe that increasing e to e' = e + afi, we can think that the true signal 
underlying the observation y is rather than w; note that (34) implies that 



< 111 + z^', v' = V + fl. 



(40) 



24 



We can now repeat the reasoning which follows (34), with (40) in the role of (34), in the role 
of w, e' in the role of e and in the role of /i, thus arriving at the following analogy of the bound 
(11): 

lk-^1|i<i±|^' + -^(e' + 5), 
v-t, 1 - C 

whence 

If - w||i < -v + -(e + d) + /i, 

1-4 1-4 

which is nothing but (12). □ 



C Proof of Proposition 4.1 



Ui = < 



Let A satisfy VSG^ (^, ^, p, cr), and let Y = [yi, .. and v satisfy (13). Let, further, / C {1, ...,n} 
be such that Card(/) < s, and let x G R". Let u € R" be given by 

'1 + 9^, i e P+ni, Xi>o 

1 - ^, i G P+ n /, Xi < 

(l + Osign(xi), iePuHl 
.0, i^I 

Note that u has at most s nonzero entries, the entries of u with indices from P+ belong to [0, 1 + 0^], 
and the modulae of entries in u with indices from P„ are < 1 + so that u^z < ^s{z) for all z. 
We have 

u''[I-Y^A]x = Y,^^C^[Y,A]x^= u^Ci[Y,A]xi+ u^[-a[Y, A]]\xi\ 



i:Xi>0 



i:Xi<0 



< ^s{Ci[Y,A])xi+ ^s{-Ci[Y,A])\xi\ [since u^z<^s{z)] 

i:xi>0 i:Xi<0 

< Yl [^ + {A^v)^]^^+ E m+{A''vUx,+ Yi^-(^''^)i]\^i\ [by (13)] 
i:Xi>0,i^P+ i:Xi>0,i£P+ i:Xi<0 



i:Xi>0,i^P^ i:Xi>0,i£P+ i:Xi<0 



+ x^A^v 



max[-Xi,^Xi] + \xi\ 

ieP+ i&Pn 



+ x^A^v, 



whence 



u^[I-Y^A\x < C 



max.[-Xi,9xi] + ^ \xi\ 

i&P+ i&Pn 



+ p\\Ax\\ 



(41) 
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(recall that < p). On the other hand, recalling the definition of u and that \\yi\\* < o", we have 



u 



^[I-Y^A], 



= u'^x — UiuJ Ax 

= Y: S,x[{l-OXr,{l + eC)xi] + {l+0 E \xi\-EUryfAx 

> Y: max[(l-C)x„(l + 0e)2:i] + (l+C) E l^il 



a 



ie/nP+ iGinPn 



\Ax\ 



<li-p 

Combining the resulting inequality with (41), we get 

[x^+im&^[-Xi,exi]] + {l + i) Y \xi\<P\\Ax\\+i 
i£inp+ ieinPn 



E 

ieP+ 



max —Xi,vxi 



+ E 

iePn 



with f3 given by (14), or, equivalently, 

5^ Xi+ E \x,\ <f3\\Ax\\+C 
ieinP+ ieinPn 



E 

iep+\i 



max —Xj,axi 



+ E 



iePn\i 



The latter relation holds true for every x G R" and for every set / C {1, ...,n} of cardinality < s, 
so that A satisfies SGs,/3(^, 6*). □ 

D Proof of Proposition 5.1 

Proof is based on the following 

Lemma D.l Let Z be a v x matrix of rank m, s > 1 be a positive integer, and 5i G (0, 1], 
1 < i < v, be such that for the columns Ci of the matrix 1^ — Z it holds ||Cj||s,i < 1 — 5j. Assume 
that 

(42) 



Then 



V > (2V2m + 1) . 



s < 2V2m + 1. 



(43) 



Proof of the lemma. Let fjj = Za, and let ji be the sum of s — 1 largest magnitudes of the 
entries in Ci with indices different from i. We have 

1 - CTj + 7i < l|Cj|ki < 1 - (^j, 

consequently ai > 5i + 7i > 0. Let us set Aj = — , and let Z be the matrix with the columns 
Zi = XiZi, where Zi is the z-th column in Z. Note that Z is of the same rank m as Z, and that 
Zii = 1 for all i. Recalling that 'ji < o'i, we have also 



— '^i||-^j||s-l,l ^ Wli + <7j] < 2Aj(7j — 2. 
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Now let s = min[s — 1, [j^^^^J], so that s > 1 due to s > 1. We have < < 2 

and < V. From the latter inequality and due to H-^iHl ^ maxjl, i/s~^}||Zj||g (cf. the proof 
of [14, Proposition 4.2]), it follows that H^jUl < 4i^s~^. We conclude that ||2'||| < 4i/^s~^, where 
for a matrix B, \\B\\2 is the Frobenius norm of B. Setting H = l[Z + Z^], we have therefore 
\\H\\2 < 4z/^s~^. On the other hand, Tt{H) = Yli=i^ii ~ while rank(//) < 2?n, whence, 
denoting by ^j, 1 < i < p < 2m, the nonzero eigenvalues of H, we have 

\\H\\l = jZ^''^ ijZ^^^)^IP = iT^^iH)f/p > ^V(2m). 

1=1 i=l 

We arrive at the inequality 4i/^s~^ > ||-ff||2 > '^^/(2?7T'), thereby 

f < 8m. (44) 

Assuming that s = [i^^^^J , (44) says that i' < {2\/2m + 1)^, which is impossible. The only other 
option is that s = s — 1, and we arrive at (43). □ 



Lemma D.l =^ Proposition 5.1: Let Y,v satisfy (13). Consider first the case when u := 
Card(P„) > n/2. Denoting by Cj the i/-dimensional vector comprised of the last i' entries in 
Ci = Ci\Y, A] (i.e., entries with indices from P„). By (13), for every i £ Pn and for every set I C Pn 
with Card(/) < s we have 

Eie/(i + e)i[c.],i < ^.(-co < e - {A^v)„ E,6/(i + om]j\ < $s(a) < c + (^^^;)i, 

thus for any i G Pn, 

2{l+mC^\\s,l<^s(.-Ci) + ^s{C^)<2^, 

SO that ||Cj||s,i < 1/2. We see that the South-Eastern u x u submatrix Z of Y^A satisfies the 
premise of Lemma D.l, while the size v of Z satisfies (42) due to (21) and > n/2. Applying the 
lemma, we arrive at (22). 

Now consider the case when Card(P„) < n/2, that is, v := Card(P+) > n/2. By (13), setting 
Ci = Ci\Y, A], for every set I C P+ with Card(/) < s and every i G P+ we have 

+ max[-[Q],-,0] < 1>.(-C0 < ^ - (A^v),, 
Ej67(l + 90 max[[Q]j,0] < $,(0 < + {A'^v),, 

whence 

Since the latter inequality holds true for every subset / of P+ with Card(/) < s, when denoting 
by Ci the part of Q comprised of the first entries (those with indexes from -P+), we have for all 
ieP+: 

\\Ci\\s,l < 1- 

Now the proof can be completed exactly as in the previous case, with the North- Western u x v 
submatrix of Y^^ A in the role of Z. □ 
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E Proof of Proposition 8.1 



Let us proceed by induction. First, let us show that (afc__i,6fc_i) implies (0^,6^). Thus, assume 
that (afc_i, bk-i) holds true. Let z^'^"^) = w — v^^~^\ By (ak-i), is supported on the support 

of w and is such that z^'^ > for i G Note that 



yj _ yik-1) _ YT{y - Av'^''-^^) = {I- Y'^A){w - v^^-^'^) - 
where e = y — Aw with ||y'^e||oo < crS due to (26. c). Then by (26.a,b) for any i G P_|_, 



E 



(TO < Z,- — Ui < T4 



E r^' + E 



j6P„ 



consequently, 

- 7- := -T-Ofc-i - < 4^""^^ - ""j < 7+ := T+afc-i + ^-i^- 
We conclude that for any i G P+ the interval = [m — 7-, + 7+] of the width 

£+ = [t_ + T+]afc„i + 2o-(5, 
covers z^'^ ^\ In the same way for any i £ Pn 

-7 := -TOk-i - (t6 < zl''~^^ - Ui < Tak-i + fT(5 = 7, 
so that the interval 5^ = [ui — j, Ui + 7] of the width 

£ = 2TQfc_i + 2a6, 

zl when i £ Pn- 

(k—l) 

Recalling that z^ > for i G P+, the closest to point of Si is 

Ai = [ui - 7]+ for i G Pn, Ui > 0, 



(45) 



covers z- 



Ai 
Ai 



[iii-7_]+ foriGP+, 
-[\ui\ - 7]+ for i G Pn, Ui < 0, 



(k~l) 

that is, Aj = Ai for all i. Since the segment Si covers z^ and Aj is the closest to point in S, 
while the width of Si is at most ^ V ^+ , we clearly have 

(a) A,GConv{0,zf-^)}, (6) {z^'^ 

Since (afc-i) is valid, (46. a) implies that 

(fc-i) , ^ /r, 

+ Conv <0,w — vl > 



Ai\ < £ V. 



(46) 



(fc) (fc-i) , , ^ 



C Conv{0, Wi}, 
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and (a/j) holds. Further, let I be the support oi w^. Relation (a^) clearly implies that \z^^^\ < \wi 
and we can write due to (46. b): 



which is (bk)- The induction step is justified. 

It remains to show that (ao,6o) holds true. Since (oq) is evident, all we need is to justify (bo) 

Let 

a* = \\w\\i, 

and let u = Y'^y. Same as above (cf. (45)), we have for all i: 

\wi — Ui\ < max{r„, T+, rja* + a5 = -a* + a5. 

s 

Then 

a* = ^ \wi\ + ^ l^il < + -a* + (t5\+ ii< ||n||s,i + pa* + scj(5 + ^l. 

iGl i0 iel 

Hence 

\\u\\s,i + saS + fi 

a* < "0 = ; , 

1 - p 

which implies (6o)- D 
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F ONLINE SUPPLEMENT 



F.l Proof of Proposition 4.2 

Let Y =[Yi, Yn],v, a, p certify the validity of VSG*^^(e, 9), and let /3' >f3,e' >e and C' G [C, !)• 
Let us set 

1 + 9^ _ 1 + g 

so that X, fJ, £ [0, 1], and let y' be as in the assertion to be proved, that is, the columns of Y' are 
multiples of those of Y: Y- = XYi when i G P+ and Y- = fiYi otherwise. All we need to prove is 
that (y , V, (7, p) certify the validity of VSG* ^/(C', d'), and this immediately reduces to verification 
of the following fact: 

Lemma F.l Let i, 1 < i < n, be fixed, and let z £ R" for any I C {1, ...,n} of cardinality s satisfy 
the relations 



< rj 



(a) [1 + 90 E max[zj-5,j,0] + (l + E \zj - 5ij\ + iAv)i < 

jeP+ni j&PnfM 

(b) (l + e^) E max[5ij-z,,0] + {l + E \zj - 5ij\ - {Av)i 

j&P+ni j£Pnni 

' iG P+, 

e, i e Pn, 



Then for every set I C {1, ...,n} of cardinality s we have 



jeP+ni jePnni 

E max[5,, -Azj,0] + (l + O E I 
jeP+n/ jePnni 

9'e, ieP+, 



(47) 





0, i/i, 


'ii = { 


1, i=j- 


(a) 




(b) 





(48) 



< 



Proof. Taking into account the definition of A, fi, in the case of z / the relations (48) are readily 
given by (47), hence we can assume i G /. Consider two possible cases: i G P+ n I and i G PnCi I- 

The case of i G P+ H /. In this case (47) reads: 

(a) (l + 0Omax[zi - 1,0] + (1 + ^0 E max[zj,0] 

jeP+ni,jj^i 

+ (1 + E \zj\ + {Av)i < ^, 
(6) (1 + eC) max[l - Zi, 0] + (1 + 0^ E max[-Zj,0] ^ ^ 

+(1 + E \zj\ - {Av)i < ec, 

jePnfM 
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and our goal is to verify that then 



(a) {l + e'^')max[Xzi-l,0] 

=1+6*5 =1+? 



+ (1 + 0'OA E max[z,-,0] + (l + O^ E \zj\ + {Av)i < ^ , 
jeP+nijy^i jePnni 

(6) (l + 0'e')max[l- Azi,0] V^Uj 
+ + 5^ max[-z„0] + (l + O Yl \zj\-iAv)i<e'e. 

^ V ' 

■.=R 

We have Xzi — 1 < A(zj — 1) due to A < 1, consequently 

max[Azj — 1,0] < max[A(zj — 1),0] = Amax[zi — 1,0], 

and therefore (50. a) follows from (49. a) due to (1 + 0'^')X = 1 + 9^ and ^' > ^. It remains to verify 
(50.6). Assume, first, that Xzi < 1. From (49.6) it follows that 

(1 + 60 [1-Zi]+R<{l + 60 max[l - Zi,0]+R<6^, 

implying Zi > and therefore 

i + 6'e~i + e'e' 

Since we are in the case 1 — Xzi > 0, we arrive at 

(1 + 0'e')max[l - Az„0] + R = (1 + 6'0)[1 - Xz,] + R < {1 + O'o ff^',^, +R = O'O, 

as required in (50.6). The case of 1 — Xzi < is trivial, since here the left hand side in (50.6) clearly 
is < the left hand side in (49.6), while 6'^,' > 6S,, so that (50.6) is readily given by (49.6). Thus, 
when i G P+ n /, (50) follows from (49). 

The case of i S Pn H /. In this case (47) means that 



(a) (1 + ^0 E max[z„0] + (l + e)|l-^i| + (l + e) E \zj\ + {Av), < 
j&p+nijy^i jePn^ijjki 

(6) (1 + ^0 E max[-z,-,0] + (l + e)|l-^i| + (l + 6 E \zj\ - {Av)i < 
jeP+ni jePnnijy^i 

and our goal is to verify that then 

(a) (l + ^'f) E max[Xzj,0] 
jeP+ni,jyti 

+(1 + ^')|1-M^i| + (1+C')/" E \zj\ + {Av)i<e, 

jdP^nij^i 

(6) {l + 6'i') E max[-Azj,0] 
ieP+n/ 

+(1 +i')\l-^,z,\ + {l + e') E \^^z,\ - {Av), < 0. 

j&Pnni,jjti 



(51) 



(52) 
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Comparing (51. o) with (52. a), and (51.6) with (52.6), we see that all we need in order to derive (52) 
from (51) is to verify the following statement: if(l+^)|l — z| < + then (1 + ^')|1 — /izj <^' + a. 
This is immediate: assuming (1 + ,^)|1 — z| < + a, the premises in the following two implication 
chains hold true: 

(l+0[l-^]<e + a^^>TTf^/^^>T^f^l-^^<l-T+F = W 

^(i + 0[i-M <e' + a, 
(i + e)[z-i]<e + a^^<i + ftf ^^^<^^^M^-i<^^ 

^ (1 + C')[^^z - 1] < 2^ - e' + a ^ (1 + C')[f^z -l]<e + a, 

while the resulting inequalities in these chains lead to the desired conclusion (1 + ^')|1 — ^z| < ^' + a. 
□ 

F.2 "Trigonometric polynomials" example 

The validity of the claim concluding Section 5 is readily given by the following 

Lemma F.2 For any positive integer d, let n > And, and A he the matrix obtained from the basic 
trigonometric polynomials as described in Section 5, then the condition VSGs(^, 0, p, o") can hold 
true for s < 2 only. 

Proof. Let L be the n x n permutation matrix corresponding to the cyclic shift ej i— t- ej^, 

= (j + 1) modn, of the standard basic orths cq, e„_i in R", and R be the m x m orthogonal 

block-diagonal matrix with the North- Western block 1 and d additional 2x2 diagonal blocks 

cos(27ri/n) — sin(27rz/n) 1 ^ . ^ , , , ^, . , c a ^ ■ ^ 

/A • / N , 1 < ^ < a. Denoting by the ?-th column oi j4, U < ? < n — 1, we 
sm(27rz/n) cos(27^^/?^) J ^ j j j , 

clearly have RAj = Aj^, hence A = RAL~^ and therefore also A = R^AL~^ for 1 < i < n. Now 
assume that Y,v satisfy (13) for certain < 1, > 1, p, a. Then 

max[^s{-Q[Y,A]) + ^s{Q[Y,A])] < ^(1 + ^), 
in this way, it is immediately seen, maxj ||Cj[i^, >1] ||s,i < k := ^j^'^^-' < 1, or, which is the same, 

r(/-y^A) < K < 1, 

where T{Z) is the maximum of the || • ||s^i-norms of columns of Z € R"^". Observe that L is a 
convex function which is symmetric in the sense that r(PZP^) = r{Z) whenever P is a permutation 
matrix. Now let F = ^ Y.7=i R~'YL\ Since L"" = /„, ii"" = 1^, we have R~^YL = Y. We claim 
that 

r(/ - Y^A) < K. 
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Indeed, we have 

1 " 

T{I-Y^A) = T{-^[I -L~'Y^R'A]) 
" 1=1 

1 

[since T is convexl 

i=l 

1 " 

= - y r(L-' \I - Y^IR'AL-']] V) 

n ^-^ 

j=i 

1 " 

= — r(/ — Y'^ A) [since V is symmetric and R^AL~^ = A] 

n ^-^ 

j=i 

= T{1-Y'^A) 



Now let 



= Ybj + ^[^2j-i,i cos(z(/)) + F2i,j sin( 



We have R~^YL = Y, that is, = YL~^. In other words, the columns Yj of y satisfy the 

relation Yj = RYj_, where j- = {j — 1) modn. This is nothing but yj^cp) = yj_{4' ~ ^ = 27r/n, 
whence yj (<?!>) = yo{4' — jS). Observe that the j-th column in Y'^A has the entries 

Yi^Aj = y,{j5) = yoiU - i)6), < i < n - 1, 

meaning that the columns in the matrix I — Y^A are cyclic shifts of each other (so that the 
II • lls^i-norms of all columns are the same), and the zero column is comprised of the values of the 
trigonometric polynomial 1 — yo{(j)) on the grid G = {(pj = : < j < n}. Assuming s > 1, 
when denoting by 7 the sum of s — 1 largest magnitudes of entries in the (n — l)-dimensional vector 
{yo(0i)}r=/, we have 

l-yo(0) + 7< l|Co[>',^]||.,i <'^< 1, 

thereby ^ := yo(0) > 7- Now let M = max |yo('/')|i and let cp G Argmaxj,|yo(</')|5 so that 

yo(^) ~ 0- -^y Bernstein theorem, we have |?/o('/')l — d'^^ for all (/>, whence |2/o(<^)| ^ M/2 when 
|(/) — <^| < \/d, so that 

71 

Card{i : |yo(<^i)| > M/2} > — - 1. 

It follows that 7 > min [s — 1, — 2] M/2, while /U = yo(0) < M. Thus, the relation ;U > 7 implies 
that 



min[s - 1, 2] < 2, 



that is, s < 2 provided that n > And. □ 



33 



